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Executive Summary 


The Impacts of Public Charter Schools on Students and Traditional Public Schools: 

What Does the Empirical Evidence Tell Us? 

The goal of this BRIEF was to compile and analyze the available evidence of the impacts 
of public charter schools on students and schools. Using a series of five common claims made 
by proponents and opponents of public charter schools as an organizer, the existing empirical 
evidence was reviewed. The analysis revealed that student performance in charter schools is 
very mixed. In some cases studies have found positive effects in achievement, others negative 
effects, and still others no effects. The presence of charter schools has resulted in positive 
changes in some traditional public schools, while in other cases there appear to be no impacts, 
either positive or negative. Public charter schools, at least those found in urban areas, tend to 
serve more disadvantaged and minority students. 

Taken in the aggregate, the empirical evidence to date leads one to conclude that we do 
not have definitive knowledge about the impacts of public charter schools on students and 
existing schools. But in reviewing the existing evidence, one is also struck by the fact that the 
impacts of charter schools appear to be very contextual. Some public charter schools are better 
than others. Some are very successful in meeting student needs, and others are not very 
successful. In other words, public charter schools vary in quality like traditional public schools. 
Success depends upon a variety of factors. Consequently, the impacts of public charter schools 
should not be painted with one broad brush stroke. Each should be judged on its own evidence 
and performance. 
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Introduction 

The goal of this BRIEF is to provide an objective review of the empirical evidence on the 
impacts of public charter schools. One of the potentially most significant reforms taking place in 
public education today is the growth in public charter schools. Since the first charter school law 
was passed in 1991 in Minnesota, the number of public charter schools has increased steadily. 
According to the National Alliance for Public Charter Schools (2013), the number of public 
charter schools nationwide has grown from approximately 1,500 in 2000 to almost 6,000 
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by 2013, a fourfold increase. Currently these 6,000 schools enroll approximately 2.3 million 
students. A majority of these schools have been established as elementary public charter 
schools, with less than 20% established as either middle or secondary schools. Over one-half 
(55%) of the public charter schools are located in large urban settings and cities, while only 
approximately 16% are located in rural areas of the country. 
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Public charter schools have both proponents and critics. Proponents often cite one set 
of research studies as evidence of their success. Critics, on the other hand, cite studies by other 
researchers as evidence of their ineffectiveness. In many cases, the research cited either by 
proponents or critics is limited to the findings from the study of only one charter school, or just 
a few schools. In other cases only charter schools in one particular urban city or charter schools 
in a particular state are studied. In some cases the research cited has been conducted or funded 
by advocacy organizations known to either support or oppose public charter schools. Only in a 
few cases have researchers attempted to develop more generalizable knowledge by examining 
the evidence across multiple states and multiple studies. 

Consequently policy makers are often confronted with what appears to be conflicting 
findings as they deliberate public charter schools policy making. Which evidence should be 
believed? Is some evidence better than others? To explore answers to these questions, a 
review of the existing empirical evidence was undertaken by Maine Education Policy Research 
Institute (MEPRI) researchers at the request of the Joint Standing Committee on Education and 
Cultural Services in the Maine State Legislature. MEPRI is a nonpartisan research institute 
charged with conducting research and analyses for the Joint Standing Committee. 

In this review every attempt has been made to identify existing evidence, and each 
study was reviewed for its design, fidelity, and findings. What follows is a summary of these 
analyses reported in terms of the some of the most common claims made in the public charter 
schools debate. These claims are: 

1. Public charter schools produce better student academic performance. 

2. Public charter schools increase competition, resulting in improvements in public 

schools. 

3. Student performance improves over time in public charter schools. 

4. Public charter schools enroll the best students from traditional public schools. 

5. Public charter schools have detrimental financial impacts on traditional public 

schools. 

Each of these claims is examined in the following pages by compiling and reviewing 
existing empirical evidence surrounding them. Although a plethora of reports and studies were 
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collected for this BRIEF, only those that were based on empirical evidence and deemed to 
reflect standard research principles and procedures were analyzed, regardless of the study 
findings. Advocacy-based reports, and reports based on anecdotal evidence alone were 
excluded. 

Claim 1: Public charter schools produce better student academic performance. 

By far the impacts of charter schools on student achievement have marshaled the most 
interest and scrutiny by proponents and critics alike. Proponents, like the Stand for Children 
national organization (2013), point to research evidence and conclude "Studies conducted by 
nationally prominent research organization show that public charter schools are successfully 
closing achievement gaps for students of color, English language learners, and students in 
poverty " (p.l). Critics like the Education Justice Organization (2013), on the other hand, look at 
the research and conclude in part that "...charter schools do not, on average, show greater 
levels of student achievement, typically measured by standardized test score, than public 
schools, and may even perform worse" (p.l). 

Which is the case? Are public charter schools more effective in improving student 
achievement or not? In reality, both the proponents and critics alike are partially correct. The 
results for public charter schools appear to be mixed. Some researchers have found that public 
charter schools improve student achievement. For example, a study of a group of charter 
schools in Texas found statistically significant positive changes in achievement for their 
students (Booker et. al., 2004). Grongberg and Jansen (2005) found similar results when they 
studied high school charter schools. And in the case of CMOs— charter management 
organizations that manage several charter schools— Furgeson et. al. (2011) found that some 
were very successful in improving student achievement. 

Flowever, there also is a body of research that shows that charter public schools have 
adverse effects on student achievement, or produce mixed results. For example. Mills (2013) 
examined 10 years of achievement data for Arkansas grades three through grade eight, 
comparing charter school students with students in traditional public schools. Fie concluded, 
"Our findings are that charter schools have small but statistically significant negative impacts on 
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student achievement for both math and literacy. Such negative effects, however, tend to 
decline with the number of years of charter operation" (p.321). Mills' findings have been 
duplicated in other quasi-experimental studies. Sass (2006) found in a study of Florida charter 
schools that charter school students experienced less gains than students in traditional public 
schools, although the gains were positive in most cases by the charter's fifth year of operation. 
Bifulco and Ladd (2006) examined 90,000+ third grade students, each over four years, and 
found that achievement of charter school children lagged behind the performance of students 
in traditional public schools. 

Why do the results from these studies appear to vary so much and produce mixed or 
conflicted findings? Some of the differences in findings can be attributed to differences in 
measures of achievement used in the various studies, or differences in types of charter schools 
studied, or differences in the demographics of students attending different charter schools. In 
light of these differences it may be useful to look for guidance in assessing the achievement 
impacts by examining studies that employ what researchers deem to be very rigorous research 
designs. Two of these designs are experimental studies and meta-analysis studies. While it is 
safe to say that more research is needed over time before we may conclude with some 
certainty about the impacts of charter schools on student achievement, these types of studies 
are considered by researchers to produce the best evidence to date on the achievement 
impacts. 

A small number of studies have been designed to examine student performance using 
an experimental design, the gold standard for research studies seeking to establish cause and 
effect. A major challenge in examining the impacts of charter schools on student performance 
is in obtaining equivalent comparison groups. Because public charter schools are schools of 
choice, it is difficult to identify equivalent groups in existing traditional school for comparisons. 
But a method used to avoid selection bias is to examine performance from over-subscribed 
public charter schools, public charter schools that were not able to admit all the students who 
applied to their school because of space limitations. As Clark et al. (2011) state in reviewing 
early studies. 
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The previous fixed effects analyses potentially provide meaningful external validity 
through the inclusion of a relatively broad geographic sample of schools, but their internal 
validity may be compromised if students attending charter schools in a given year differ 
from those who do not in ways that are not fully captured by the fixed effects models. In 
contrast, the lottery-based studies potentially provide strong internal validity by comparing 
lottery applicants who were randomly admitted to charter schools to those who were not 
(P-1) 


From a research point of view, the lottery-based studies provide the opportunity to compare 
two equivalent groups using an experimental design with the only difference in the students 
being whether they were admitted or not admitted to the charter school. 

Four experimental studies are particularly noteworthy. The first was a 2005 study by 
Floxby and Rockoff of three charter schools in Chicago. In the words of the researchers, 

1/1/e use the “lotteried- out" students as a control group for the "lotteried-in" students 
because randomization makes the groups similar on unobservable characteristics, such as 
motivation, as well as on observable traits, such as race and prior achievement. 1/1/e 
estimate both the effect of attending charter schools (the treatment-on-the-treated effect) 
and the effect of being offered the chance to attend a charter school (the intent-to-treat 
effect). 1/1/e show that, compared to their lotteried-out fellow applicants, students who apply 
to and attend charter schools starting in the elementary grades score about six national 
percentile rank points higher in both math and reading. These effects are for students who 
have spent an average of two years in charter school (p. 1). 

Thus, the researchers found that students enrolled in the Chicago charter schools in the 
study performed 5-6 percentile points better in reading and mathematics than their 
non-admitted cohorts. 


A second study by Floxby and his colleagues (Floxby, Muraka, & King; 2009) of a larger 
number of urban schools in New York City found that upper elementary grade students (grades 
4-8) in public charter schools outperformed similar grade students who were not admitted to 
the charter school. And a third study of oversubscribed charter schools in Boston found that 
students admitted to the charter schools performed better than students not admitted to the 
schools by approximately one-quarter standard deviation. After examining student 
achievement data for students in grades 4-8 and grade 10 in oversubscribed charter schools. 
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the researchers concluded. 


On balance, our lottery-based findings provide strong evidence that the charter model has 
generated substantial test score gains in high-demand Charter Schools with complete 
records. On the other hand, these results should not be interpreted as showing that Boston 
Charters always produce test score gains (p.39). 

A fourth study, and one potentially producing more generalizable findings, included 36 
middle schools across 15 states. The study compared the two-year performance of over 1,700 
middle school students who were admitted to charter schools to over 1,100 who applied but 
were not admitted to these charter schools. The researchers reported that: 

1/1/e found that, on average, the charter schools in the study had an insignificant or 
negative impact on student achievement in reading and math. Impacts generally did not 
vary across subgroups defined by students’ race, or gender. However, impacts were 
insignificant or positive for more disadvantaged students and negative for more 
advantaged students, and this same pattern persisted across groups defined by baseline 
test scores. There was also considerable variation in impacts across schools. Those in 
urban areas or serving more disadvantaged populations had more positive (or less 
negative) impacts than those in non-urban areas or serving more advantaged populations 
(P-23). 

Thus, the evidence from experimental studies of the impacts of public charter schools 
appears to provide mixed results. Some found positive effects on student achievement, while a 
multi-state study reported positive results for some disadvantaged students and negative 
results for some advantaged students. 

A second type of more rigorous research design is called a meta-analysis, which uses a 
technique involving examining results across multiple studies using a standard statistical 
procedure. Traditionally, the technique of meta-analysis has involved reviewing research 
studies on a topic and counting the number of studies that found positive results and the 
number that found negative results. The conclusion reached from the meta-analysis was in 
favor of whichever group of studies was larger. This technique is still used, but a more 
sophisticated methodology has been used in current meta-analyses, one that involves 
conducting statistical analyses using the individual empirical results from multiple studies. All 
three of the meta-analyses reported here used both methodologies in their analyses. 

The meta-analyses examined for this BRIEF also produce mixed results. The three were 
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selected for review because they are considered to be rigorous, unbiased studies. The first 
meta-analysis was conducted by Betts and Tang (2011) for the National Charter School 
Research Project (NCSRP) researchers from the University of Washington Center on Reinventing 
Public Education (CRPE). Researchers examined the value-added effects of charter schools by 
analyzing 25 empirical studies, some of which have been described above. The researchers 
found overall effect sizes in the range of 0.020-0.055 in favor of charter school elementary and 
middle students reading and mathematics achievement. Effect sizes are standard deviation 
units, with positive effect sizes denoting positive results. These types of results will be discussed 
more in subsequent paragraphs, but in the case of this meta-analysis the researchers concluded 
in part that: 

The overall tenor of our results is that charter schools are in some cases 
outperforming traditional public schools in terms of students’ reading and math 
achievement, and in other cases performing similarly or worse. There are several 
important cases of grade spans in which charter schools are outperforming or performing 
about as well as traditional public schools. Elementary school math and reading, middle 
school math and, only if we include the KIPP school estimates, middle school reading all 
exhibit this pattern of students performing better at charter schools than at traditional public 
schools. At the high school level, there is no overall significant effect of charter schools, 
but there is considerable heterogeneity, suggesting that in some locations charter high 
schools are outperforming, while in others they are underperforming (p.55). 

The second and third meta-analyses were conducted by the same organization, the 
Center for Research on Education Outcomes (CREDO) at Stanford University. The first study, 
reported in 2009, analyzed charter school performance in 16 states, states that at the time of 
the analysis enrolled 70% of the charter school children nationwide. The overarching conclusion 
from the meta-analysis was that student performance was not better in charter schools. Using 
the traditional counting method of analyses, the researchers found: 

The group portrait shows wide variation in performance. The study reveals that 
a decent fraction of charter schools, 1 7 percent, provide superior education 
opportunities for their students. Nearly half of the charter schools nationwide have 
results that are no different from the local public school options and over a third, 37 
percent, deliver learning results that are significantly worse than their student would 
have realized had they remained in traditional public schools. These findings underlie 
the parallel findings o f significant state -by state differences in charter school 
performance and in the national aggregate performance of charter schools. The policy 
challenge is how to deal constructively with varying levels of performance today and 
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into the future. 


Using the more sophisticated meta-analysis technique, the researchers found: 


The national pooled analysis of charter school impacts showed the following 

results: 

Charter school students on average see a decrease in their academic growth in reading of 
.01 standard deviations compared to their traditional school peers. In math, their learning 
lags by .03 standard deviations on average. While the magnitude of these effects is small, 
they are both statistically significant. 

The effects for charter school students are consistent across the spectrum of starting 
positions. In reading, charter school learning gains are smaller for all students but those 
whose starting scores are in the lowest or highest deciles. For math, the effect is 
consistent across the entire range. 

Charter students in elementary and middle school grades have significantly higher rates of 
learning than their peers in traditional public schools, but students in charter high schools 
and charter multi - level schools have significantly worse results. 

Charter schools have different impacts on students based on their family backgrounds. For 
Blacks and Hispanics, their learning gains are significantly worse than that of their 
traditional school twins. However, charter schools are found to have better academic 
growth results for students in poverty. English Language Learners realize significantly 
better learning gains in charter schools. Students in Special Education programs have 
about the same outcomes. 

Students do better in charter schools over time. First year charter students on average 
experience a decline in learning, which may reflect a combination of mobility effects and 
the experience of a charter school in its early years. Second and third years in charter 
schools see a significant reversal to positive gains. 

Thus, the overall findings from this 2009 analysis indicate that the charter schools were 
not having positive impacts on student performance, and in some cases negative effects. It 
should be noted that some individual studies found positive gains, but when analyzed in the 
aggregate, the results were not positive. 

In 2013 the same organization at Stanford University repeated their earlier analysis 
using student performance data from charter and traditional schools in 25 states. In this second 
meta-analysis, the researchers found small but statistically significant positive effects in favor of 
charter schools. The analysis of performance at the student level revealed 0.01 standard 
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deviation improvements in reading when comparing the performance of students in charter 
school with those in traditional public schools. Performance in mathematics was approximately 
the same in both types of schools, public charter schools and traditional public schools, but 
mathematics performance did improve from the 2009 study. Accordingly, the researchers 
concluded: 


In the aggregate, both reading and math results show improvement compared to 
the results reported in Multiple Choice. The analysis of the pooled 27 states shows that 
charter schools now advance the learning gains of their students’ more than traditional 
public schools in reading. Improvement is seen in the academic growth of charter students 
in math, which is now comparable to the learning gains in traditional public schools. On 
average, students attending charter schools have eight additional days of learning in 
reading and the same days of learning in math per year compared to their peers in 
traditional public schools. In both subjects, the trend since 2009 is on an upward trajectory, 
with the relative performance of the charter sector improving each year. Related results for 
different student groups indicate that black students, students in poverty, and English 
language learners benefit from attending charter schools. However, charter school quality 
is uneven across the states and across schools (p.3). 

Thus, based on the most recent meta-analysis the researchers have concluded that 
charter schools are showing positive effects on student learning. And even though most 
reviewers of these research studies would agree that they were well done, there are two 
important concepts to be kept in mind when formulating conclusions based on the findings in 
these studies. These two concepts are statistical significance and effect size. 

Probability theory underlies statistical analyses of research evidence and is used to 
determine if a finding is statistically significant. Statistical significance testing is a statistical 
procedure applied to results from studies to determine if the results are by chance or if they 
represent real changes or difference. In other words, just because there is a difference in 
performance— in this case between students in charter schools and those in traditional public 
schools— the results might still be by chance. The test of statistical significance analyzes the 
probability that the results are by chance. If the probability is below 5% then the researcher 
may conclude that a difference is real, and not just chance. 

Whether the probability in any particular case is below 5% depends in part on sample 
sizes. Larger sample sizes increase the chance the researcher will find the probability to be less 
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than 0.05. In all three of the meta-analysis studies described above, the sample sizes were very 
large and the findings were statistically significant. However, the actual differences in 
performance between public charter school students and traditional public school students 
were very small. Because the sample sizes were very large, the statistical analysis was able to 
detect small differences that may have been disregarded as random chance if the sample size 
were smaller. Consequently, sample sizes may have contributed to finding statistically 
significant results. 

This artifact of research using large samples highlights the importance of the second key 
concept. Effect Size . A result may be statistically significant, but is it of practical significance? 
Should policy or programs be changed based on the research results? To answer this type of 
question, researchers have created a way to measure practical significance. It is called Effect 
Size. Instead of measuring differences based on sample size. Effect Size captures the magnitude 
of differences and this difference is measured in terms of standard deviations. Larger standard 
deviations would translate into greater levels of practical significance. Cohen (1969) deems 
Effect Sizes below .20 to be small in terms of practical significance. 

A re-examination of the Effect Sizes in these meta-analyses reveals very small 
differences in standard deviations. For example, in the most recent Stanford University study, 
the differences in performance between charter school student performance and student 
performance in traditional public schools were found to be 0.01 in reading. This means that 
reading performance of charter school students was one-percent of a standard deviation better 
than reading performance of other students in traditional public schools. This Effect Size is far 
below the .20 Effect Size Cohen characterized as being small. To help translate what this means, 
the Stanford researchers converted this difference into days of learning. They concluded that 
the 0.01 standard deviation in reading translates into approximately 7 days of learning each 
school year. So applying this conversion metric to the two CREDO studies, the 2009 results 
show that learning of charter school students is seven days behind traditional public school 
students, whereas in 2013 the charter school students are seven days ahead in learning 
compared to their classmates in traditional public schools. 

Thus, while all three meta-analyses report statistically significant changes (some positive 
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and some negative), the practical importance of these changes is small in all cases. This does 
not mean these studies should be discounted as not being important, but rather that it might 
be better to exercise a degree of caution in attaching meaning to the results, at least these 
results from the meta-analyses. 

The same may be said for the evidence on the effectiveness of virtual charter schools. 
Molar (2013) describes virtual education in the following way: 

Virtual education takes many forms and serves many purposes. Formats include full-time 
online K-12 schools as well as single courses that allow students to explore a subject not 
available in their brick-and-mortar schools. Virtual education is also sometimes used by 
students to make up credits for a required course they earlier failed. Some virtual 
education programs require students and teachers to be online at the same time 
(synchronous education); others allow students and teachers to visit online courses at their 
own convenience (asynchronous education). Others combine online work with traditional, 
in-person classroom instruction (blended instruction). Providers include public entities, 
non-profit and for-profit organizations (p. i). 

Recently these type of virtual schools have received considerable attention and press 
coverage, and quite a bit of it has been unfavorable. However, it is important to note that there 
has been little rigorous research conducted on the impacts of these charter schools on student 
achievement. After reviewing studies conducted over a decade, a U.S. Department of 
Education study (2010) report concluded: 

Few rigorous research studies of the effectiveness of online learning for K-12 students 
have been published. [Italics in original.] A systematic search of the research literature 
from 1994 through 2006 found no experimental or controlled quasi-experimental studies 
comparing the learning effects of online versus face- to-face instruction for K-12 students 
that provide sufficient data to compute an effect size (p.xiv). 

A more recent study conducted by CREDO (2011) of the virtual charter schools in 
Pennsylvania found that student performance in these schools was lower than for their 
classmates in traditional public schools. Based on an examination of Pennsylvania charter 
schools over a four year period, the researchers determined: 

Two types of charter schools are authorized in Pennsylvania: physical brick and mortar 
schools and cyber, or virtual, schools. The student populations at the two types of schools 
differ. The typical cyber charter student is white and ineligible for subsidized meals, while 
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the typical brick and mortar charter student is black and receiving free or reduced-priced 
lunches. Furthermore, the starting score for cyber students is significantly higher than for 
brick and mortar charter students in both reading and math. Additionally, cyber students 
are more likely to be repeating a grade than brick and mortar charter students. ...The 
learning gains for students in brick and mortar charter schools in Pennsylvania were not 
significantly different from their traditional public school counterparts in reading. Brick and 
mortar charter students learned significantly less on average than their counterparts in 
math. Cyber charter students have significantly smaller gains in reading and math than 
those of their traditional public school peers. 

In addition, and given that students in virtual school are typically different from a 
majority of students attending traditional public schools, these researchers matched students 
in the two types of schools and analyzed their academic performance. They found that: 

In every subgroup with significant effects, cyber charter performance is lower than the 
brick and mortar performance. English Learner students at both types of charter schools 
have similar learning gains to fluent speakers in traditional public schools in math (p. 10). 

But aside from this analysis of Pennsylvania virtual schools, far too little rigorous 

research has been conducted on virtual charter schools. A recently published meta-analysis of 

virtual charter schools by the National Education Policy Center (2013) concludes: 

In short, given the results of the few rigorous K- 12 studies that have been done, there is 
insufficient evidence for policymakers to promote major online initiatives in either 
elementary or secondary schools (p.62). 

Virtual charter schools, while experiencing considerable growth in numbers in recent 
year, are still in need of rigorous study to determine their impacts on student learning. 

In summary, the evidence on the impacts of charter schools on student learning is 
neither clear nor conclusive. Some studies point to positive results while others have reported 
mixed or negative results. And if these results are combined with those found in meta-analysis 
studies, it may be wise to conclude that the jury is still out on the impacts of public charter 
schools on student performance. 

Claim 2: Public charter schools increase competition resulting in improvements in 
public school performance. 

Proponents of public charter schools often put forth the claim that the existence of 
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charter schools increases competition. Traditional public schools will feel compelled to modify 
their programs and structures, or add new programs to better respond to their student needs. 
As a result student performance will improve in these non-charter schools. However, a review 
of existing empirical evidence reveals the results are mixed in this area. Some results supported 
this claim. For example, a study of Florida charter schools uncovered a small positive impact in 
student achievement for students in traditional public schools (Sass, 2006). After examining 
student performance over a five year period of time, Sass concluded, "...competition from 
charter schools is associated with modest increases in math scores and unchanged reading 
scores in nearby traditional public schools" (p.91). Similar results were found by Hoxby in a 
Michigan study (2003). After examining evidence over a four year period of time, Hoxby 
concluded, "Overall, the picture that one draws from Michigan is the following. Public schools 
that were subjected to charter competition raised their productivity and achievement in 
response, not only exceeding their own previous performance but also improving relative to 
other Michigan schools not subject to charter competition" (p.333). 

In contrast, another Michigan study found no significant improvements (Bettinger, 
2005). After comparing test scores for students attending charter schools and those attending 
public schools, Bettinger concluded, "The results suggest that charter schools have had no 
significant effect on test scores in neighboring public schools" (p.133). And negative effects 
were found in an Ohio study (Carr & Ritter, 2007). Using a time series design to examine four 
years of achievement data, the researchers concluded, "...that charter school competition has a 
consistently small but significant negative effect on the proficiency passage rates of nearby 
traditional public schools" (p.l). 

One possible reason for these mixed results may be related to how much the traditional 
schools change the way they conduct business. A study by Holly, Egalite, and Lueken (2013) 
reports that the presence of public charter schools in the urban districts they considered had 
spurred competition and innovation in traditional public schools. However, the data used in the 
study consisted of news reports and website minutes of board meetings, not what would be 
considered a rigorous examination of the evidence. In a more rigorous study of public charter 
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schools in Massachusetts, New Jersey, and Washington D.C. researchers concluded, "We found 
that charter competition has not induced large changes in district-wide operations, despite the 
fact that a significant number of students have left district schools" (Teske, Schneider, Buckley 
& Clark, 2000, p.l). In another study, Arsen and Ni (2012) found that their analysis of statewide 
data on Michigan school districts did not support the hypothesis that the presence of public 
charter schools resulted in the reallocation of district resources toward more achievement- 
oriented activities. 

Thus, it is still unclear whether charter schools create enough competition to force 
traditional schools to change practices. After reviewing the existing evidence, what Arsen and 
Ni concluded in 2008 appears to be still true today. They said, "We find that results from 
available empirical studies are mixed and do not yet allow for firm conclusions about the effects 
of competition on traditional schools and non-choosing students" (p.l). 

Claim 3: Student performance in public charter schools improves over time. 

The National Alliance for Public Charter Schools reports that approximately 150 public 
charter schools closed in 2011-12. Closures resulted for a variety of reason, including low 
enrollments and financial conditions, but a major reason was low student performance. Many 
proponents of public charter schools argue that while some schools may struggle in the early 
years of their existence that performance improves over time. In some cases this is true, but 
the empirical evidence suggest that this is not necessarily the normal trend for public charter 
schools. One five-state analysis reported that the so-called list of "High Fliers" and "Bottom 
Dwellers" in terms of school performance did not change over a five year period of time 
(Fordham, 2013). The analysis of charter schools in the five cities of Albany, Chicago, Cleveland, 
Denver and Indianapolis revealed that charter schools performing well tended to continue to 
perform well while charter schools performing poorly tended to continue to perform poorly. 
Results from the major 2013 CREDO study discussed earlier corroborate these findings. 

Another part of their analysis was to examine how student performance changes over time in 
new and continuing public charter schools. The results revealed three key findings. First, 
"...there appears to be no structural 'new school' phenomenon of wobbly performance for 
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several years" (p.5). Some public charter schools produce good student performance from the 
first day of enrollment while other struggle and some fail to improve. Second, the analysis 
revealed that the best trajectory for improved performance was found in schools that grew one 
grade at a time instead of multiple grades all at once. 


Third, to examine if performance improves over time, the researchers divided the 
schools in quintiles in terms of student performance. Quintile 1 included schools that had low 
student performance. Quintile 5 included schools with the highest performance. The 
researchers concluded that: 

Taking the first available performance measure and using it to predict one-year 
increments going forward, 80 percent of schools in the bottom quintiles of performance 
remain low performers through their fifth year. Additionally, 94 percent of schools that 
begin in the top quintile remain there over time. 

If we wait until the third year to start the predictions (i.e. use two growth periods as 
the basis of setting the initial performance for the subsequent conditional probabilities), the 
patterns are even stronger: 89 percent of low performing schools remain low performing 
and 97 percent of all the high flyers persist at the top of the distribution. 

Only the schools in the 2 nd quintile show any substantial pattern of movement, with 
half of the schools moving to a higher quintile (mostly to the 3 rd ) and half remaining in the 
bottom two quintiles. 

Thus, the evidence indicates that some public charter schools can perform well 
even in their first year of existence, and that the first year is a very good indicator of 
how the school will perform over time. Higher performing schools remain higher 
performing, and lower performing schools remain lower performing. Most assuredly 
there are exceptions to this general rule, but in a majority of cases first year 
performance appears to be a good barometer of later performance. 


Claim 4: Public charter schools enroll the best students from traditional public schools. 

Opponents of public charter schools often argue that charter schools take the best 
students from traditional public schools. In other words, they "cream the best crop" of 
students, leaving more disadvantaged students in the traditional public schools. While it 
appears this perception is fairly widespread, the empirical evidence does not support this claim. 
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Cowen and Winters (2013) conducted an analysis of five years of data on Florida students who 
had moved to charter schools. They concluded: 

The better students are performing relative to their peers, the less iikeiy they are to 
move into a charter. There are higher rates of minority students in the charter sector, and 
Hispanics appear especially likely to transfer. On the other hand, special-needs students 
are less likely to be in charters. White students are more iikeiy to enter a charter school 
with more white students than minority students who are more likely to move into charters 
with more minority students. These findings suggest that charter schools do not “cream 
skim ” the best students away from the traditional sector, although there is evidence that 
charters do not provide an educational home for all.” (p. 2 10) 

Cowen and Winters' findings from Florida appear to mirror those found nationwide. A 
majority of public charter schools nationwide enroll more disadvantaged and minority students 
that traditional schools. According to the National Alliance for Public Charter Schools (2013), in 
2010-2011 "... public charter schools across the nation enroll, on average, a greater percentage 
of low income students (46 percent versus 41 percent). Black and Latino students (27 percent 
versus 15 percent and 26 percent versus 22 percent, respectively), and students who perform 
lower on standardized assessments before transferring to charter schools "(pi). 

The 2013 CREDO study by Stanford found similar student demographics for public 
charter school students. Their analysis of student demographic data across 27 states found that 
a majority of students in charter schools are students in poverty, as defined as qualifying for 
free or reduced lunches. Typically students in poverty struggle more in terms of achievement. 
Additionally, they found that in the charter schools they studied across the country, a slightly 
higher percent of the students were special needs students and English Language learners. 

The flip side to the selection bias argument is the argument that charter schools "push 
out" lower performing students. That is, charter schools counsel out students who are not 
performing well in the charter school in order to improve their overall school academic 
performance and at the same time reduce the costs of educating more challenging students. 
Few researchers have systematically examined if this phenomena exists, but a recent study by 
Zimmer and Guarino (2013) provides some evidence that this does not appear to be happening 
in at least one urban district. The researchers studied the movement of students in a large 
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(anonymous) urban school district over a seven year period of time. From their analysis of 
student demographic and academic performance data, the researchers concluded: 

[ 0]ur analysis suggests that there is no evidence consistent with the claim that charter 
schools are in general or at the individual level pushing out low-performing students". 

Although there needs to be more research in other districts or states, our results weaken 
the ‘push out’ argument against the establishment of charter schools in general (p.476). 

However, it is important to recognize that this is only one study. Although it is 
well done, from a methodological point of view, more research is needed in this area 
before patterns may be apparent. 

Thus, a majority of students enrolled in charter schools are not the higher performing 
students. Quite to the contrary, they are more likely to be members of minorities and children 
in poverty, and some have special needs or English language challenges. These student 
demographics are not too surprising in light of two other pieces of information. Many charter 
schools are established specifically to address needs of some students. As the National Alliance 
for Public Charter Schools reports (2013), "Many charter operators make the strategic decision 
to open charter schools in underserved neighborhoods with high concentrations of low income, 
minority, and low performing students" (p.l). 

In addition, the public policy laws governing public charter schools in most states 
prohibit selectivity in student enrollments, and most employ a lottery system to govern (at least 
in part) student enrollments. However, it is also important to recognize that a majority of public 
charter schools are found in urban settings. The Stanford analysis revealed that over one-half of 
all the public charter schools they studied across the 27 states were in urban areas, and only 
16% were in rural settings. 

To date there is very little reliable information specifically on the student demographics 
and academic performance of students enrolled in more rural public charter schools. An 
examination of the student performance data in some studies suggest many students in rural 
charter schools also struggle academically, but it not known at this time if the student 
demographics in more rural public charter schools mirror the student demographics in urban 
charter schools. Thus, while evidence suggests that traditional public schools do not experience 
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an exodus of higher performing students to public charter schools, this finding does come with 
a caveat. Less information is known about the performance of students leaving traditional rural 
public schools for public charter schools. 

Claim 5: Public charter schools have detrimental financial impacts on sending public 
schools. 

Many opponents of public charter schools argue that charter schools have detrimental 
effects on the traditional public schools that students are leaving to enroll in public charter 
schools, and that this may negatively affect the remaining students in the traditional school. 

The empirical evidence for this claim appears to be mixed and very localized. That is to say, 
some studies have reported negative effects in particular cities and particular states. For 
example, a study of public charter schools and traditional public schools in one county in North 
Carolina found that enrollments had declined in the traditional public schools while per pupil 
expenditure levels were affected in only some of the schools (Cho et a I, 2013). The study by 
Bifulco and Reback (2011) of Albany and Buffalo, two cities considered by the researchers to 
have high concentrations of charter schools, found "...that charter schools have had negative 
fiscal impacts on these two school districts..." (p.l). They estimated that the negative fiscal 
impacts ranged from $633-$l,070 in per pupil expenditures for traditional public schools, 
depending upon city. 

An Ohio study, using data provided by the Ohio Department of Education, found that 
funding public charter schools resulted in a 6.5% loss in state aid for students in traditional 
public schools (Innovation Ohio, 2013). The study by Teske and others (Teske et al, 2000) of 
schools in two states and the District of Columbia, albeit urban school districts, found that most 
districts did not see reductions in expenditures, in part because of strategies implemented for 
increasing their enrollments. And the study of Michigan schools, by Arsen and Ni (2012) 
reported that public charter schools did, "...clearly generate fiscal stress in districts" (p.3). 

Thus, the evidence to date of the impacts of public charter schools on traditional public 
schools is at best unclear. At least in urban settings, the presence of public charter schools does 
not appear to have major impacts on student demographics in traditional public schools. 
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However, in the area of potential financial impacts, the existence of public charter schools does 
appear to result in lower per pupil expenditures in traditional public schools. Additional 
research is needed on the potential impacts on rural traditional public schools, but it is likely to 
uncover similar financial impacts, given the strong relationship between student enrollments 
and expenditures found in most states. 

Summary 

The goal of this BRIEF was to compile and analyze the empirical evidence of the impacts 
of public charter schools on students and schools. The analysis revealed that student 
performance in charter schools is very mixed. In some cases studies have found positive effects 
in achievement, others negative effects, and still others no effects. The presence of charter 
schools have resulted in positive changes in some neighboring traditional public schools, while 
in other cases there appear to be negative impacts, or no impacts (either positive or negative). 
Public charters schools, at least those found in urban areas, tend to serve more disadvantaged 
and minority students. 

Taken in the aggregate, the empirical evidence to date leads one to conclude that we do 
not have definitive knowledge about the impacts of public charter schools on students and 
schools. But in reviewing the existing evidence, one is also struck by the fact that the impacts of 
charter schools appear to be very contextual. Some public charter schools are better than 
others. Some are very successful in meeting student needs, and others are not very successful. 
In other words, public charter schools are just like traditional public schools. Success depends 
upon a variety of factors. Consequently, the impacts of public charter schools should not be 
painted with one broad brush stroke. Each should be judged on its own evidence and 
performance. 


19 



20 



References 


Abdulkadiroglu, A., Angrist, J., Cohodes, S., Dynarski, S., Fullerton, J., Kane, T., & Pathak, P. 
(2009), Informing the Debate: Comparing Boston's Charter, Pilot and Traditional Schools. 
Boston, MA: The Boston Foundation. 

Arsen, D., & Ni, Y. (2012). The Effects of Charter School Competition on School District Resource 
Allocation. Educational Administration Quarterly, Vol. 43, No. 3., p. 3-37. 

Arsen, D., & Ni, Y. (2008). The Competitive Effect of School Choice Policies on Performance in 
Traditional Public Schools. Retrieved from 

http://www.greatlakescenter.org/docs/Research/2008charter/policy briefs709.pdf . 

Bettinger, E. P. (2005). The Effect of Charter Schools on Charter Student and Public Schools. 
Economics of Education Review, Vol. 24, No. 2, p.133-147. 

Betts, J.R., & Tang, Y.E. (2011). The Effects of Charter Schools on Student Achievement: A 
Meta— Analysis of the Literature. Seattle, WA: National Charter Research Project, University of 
Washington. 

Bifulco, R., & Ladd, H.F. (2006). "The Impact of Charter Schools on Student Achievement: 
Evidence from North Carolina." Education Finance and Policy, vol. 1, no. 1, pp. 50-90. 

Bifulco, R, & Reback, R. (2011). Fiscal Impacts of Charter Schools: Lessons from New York. 
Albany, NY: Research Foundation, State University of New York. 

Booker,K., Gilpatric, S.M., Gronberg, T., & Jansen, D. (2004). Charter School Performance in 
Texas. Knoxville, TN: University of Tennessee. 

Carr, M. & Ritter, G. (2007) Measuring the Competitive Effect of Charter Schools on Student 
Achievement in Ohio's Traditional Public Schools. Fayetteville, AR.: University of Arkansas. 

Center for Research on Education Outcomes (2009). "Multiple Choice: Charter School 
Performance in 16 States." Stanford, CA: CREDO, Stanford University. 

Center for Research on Education Outcomes (2011) "Charter School Performance in 
Pennsylvania". Stanford, CA: Stanford University. 

Center for Research on Education Outcomes (2013)"Charter School Growth and Replication," 
Stanford, CA: CREDO, Stanford University. 

Cho, E., Chudnofsky, A., Jian, Y., Landes, M., & Mortimer, L. (2013). The Impact of Charter 
Schools on Traditional Public School Finances and Demographics. Durham, NC: Duke University. 

Clark, M.A., Gleason, P., Tuttle, C.C. (2011). Do Charter Schools Improve Student Achievement? 
Evidence from a National Randomized Study. Princeton, NJ: Mathematica Policy Research. 


21 


Cohen, J. (1969). Statistical Power Analysis for the Behavioral Sciences. NY: Academic Press. 

Cowen, J.M., & Winters, M.A. (2013). "Choosing Charters: Who Leaves Public School as an 
Alternative Sector Expands?" Journal of Education Finance, Vol. 38, Number 3, pp. 210-229. 

Education Justice (2013). Charter School Achievement: Hype vs. Evidence. Retrieved from 
http://www.educationjustice.org/nlej_iss21_art5. 

Facts on Public Charters (2013). Washington, DC: Stand for Children. Retrieved from 

http://www.stand.org/evidenceoncharterschools . 

Furgeson, J., et al. (2011). Charter-school Management Organizations: Diverse Strategies and 
Diverse Student Impacts. Princeton, NJ: Mathematica Policy Research. 

Gronberg, T. J., & Jansen. D.W. (2005). Texas Charter Schools: An Assessment In 2005. Texas 
Public Policy Foundation, http://www.texaspolicv.com/pdf/2005-09-charterschools-rr.pdf . 

Holley, M.J., Egalite, A.J., & Lueken, M.F. (2013). Competition with Charters Motivates Districts. 
Retrieved from http://www.educationnext.org/iournal/fall-2013-vol-13-no-4 . 

Hoxby, C.M. (2003). School Choice and School Productivity, Could School Choice Be a Tide that 
Lifts All Boats? In Hoxby, C.M. (Ed.)(2003). The Economics of Choice. Washington DC, National 
Bureau of economic Research, University of Chicago Press. 

Hoxby, C. M., & Rockoff, J.E. (2005). "Findings from the City of Big Shoulders." Education Next, 
vol. 5, no. 4, pp. 52-58. 

Hoxby, C.M., Murarka, S, & King, J. (2009). "How New York City's Charter Schools Affect Student 
Achievement: August 2009 Report." Second report in series. Cambridge, MA: New York City 
Charter Schools Evaluation Project. 

Innovation Ohio (2013). Unfair Funding: How Charter Schools Win & Traditional Schools Lose. 
Retrieved from http://innovationohio.org/wp-content/uploads/2013/02/Unfair-Funding-IQ- 
charter-schools-report-l.pdf 

Means, B. et. al. (2010). Evaluation of Evidence-Based Practices in Online Learning: A Meta- 
Analysis and Review of Online Learning Studies. Washington, DC: Office of Planning, Evaluation, 
and Policy Development, U.S. Department of Education, xiv. Retrieved from 

http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf 

Mills, J.N. (2013). The Achievement Impacts of Arkansas open-Enrollment Charter Schools. 
Journal of Education Finance, Vol. 38, No. 4. 

Molnar, A. (Ed.) (2013). Virtual Schools in the U.S. 2013: Politics, Performance, Policy, and 
Research Evidence. Boulder, CO: National Education Policy Center, University of Colorado. 


22 


National Alliance for Public Charter Schools (2013). Back to School Tallies: Estimated Number of 
Public Charter Schools & Students, 2012-2013. Retrieved from 

http://www.publiccharters.org/data/files/Publication doc 

National Alliance for Public charter Schools (2012). Details from the Dashboard: Charter School 
Race/Ethnicity Demographics. Retrieved from 

http://www.publiccharters.org/data/files/Publication docs 

Pubic Impact and Fordham Institute (2013). Searching for Excellence: A Five-city Cross-state 
Comparison of Charter School Quality. New York: Thomas B. Fordham Institute. 

Sass, T. R. "Charter Schools and Student Achievement in Florida." Education Finance and Policy, 
vol. 1, no. 1, winter 2006, pp. 91-122. 

Teske, P., Schneider, M. Buckley, J., & Clark, S. (2000). Does Charter School competition 
Improve Traditional Public Schools? Civic Report. Retrieved from http://www.manhattan- 
institute.org/html/cr 10 . 

U.S. Department of Education, National Center for Education Statistics. (2013 ). The Condition of 
Education 2013 (NCES 2013-037), Charter School Enrollment. 

Zimmer, R.W., & Guarino, C.M. (2013). Is There Empirical evidence That Charter Schools "Push 
Out" Low-Performing Students? Educational Evaluation and Policy Analysis, vol. 35, No. 4, 
pp. 461-480. 


23 


